Systems, methods, and devices for pipelined processing of online advertising performance data

ABSTRACT

Systems, methods, and apparatus are disclosed for identifying and analyzing online advertising performance data. Systems may receive data records that include data events characterizing interactions between users and online advertisement campaigns. The online advertising data records may include timestamp data characterizing creation dates for data events. The systems may generate intermediate data objects by partitioning at least some of the online advertising data records based on a first plurality of temporal data categories representing different units of time. Each intermediate data object may be associated with a temporal data category of the first plurality of temporal data categories. The systems may generate performance data objects by partitioning the intermediate data objects based on a second plurality of temporal data categories that are different than the first temporal data categories. Each performance data object may be associated with a temporal data category of the second plurality of temporal data categories.

TECHNICAL FIELD

This disclosure generally relates to online advertising, and morespecifically to identifying and analyzing performance data associatedwith online advertising.

BACKGROUND

In online advertising, internet users are presented with advertisementsas they browse the internet using a web browser or mobile application.Online advertising is an efficient way for advertisers to conveyadvertising information to potential purchasers of goods and services.It is also an efficient tool for non-profit/political organizations toincrease the awareness in a target group of people. The presentation ofan advertisement to a single internet user is referred to as an adimpression.

Billions of display ad impressions are purchased on a daily basisthrough public auctions hosted by real time bidding (RTB) exchanges. Inmany instances, a decision by an advertiser regarding whether to submita bid for a selected RTB ad request is made in milliseconds. Advertisersoften try to buy a set of ad impressions to reach as many targeted usersas possible. Advertisers may seek an advertiser-specific action fromadvertisement viewers. For instance, an advertiser may seek to have anadvertisement viewer purchase a product, fill out a form, sign up fore-mails, and/or perform some other type of action. An action desired bythe advertiser may also be referred to as a conversion.

SUMMARY

Systems, methods, and devices are disclosed herein for identifying andanalyzing performance data associated with online advertising. Systemsmay include a first processing node configured to receive a plurality ofonline advertising data records, the plurality of online advertisingdata records including a plurality of data events characterizing aplurality of interactions between at least one user and an onlineadvertisement campaign, and the plurality of online advertising datarecords including timestamp data characterizing a plurality of creationdates associated with the plurality of data events. The systems mayfurther include a second processing node configured to generate aplurality of intermediate data objects by partitioning at least some ofthe plurality of online advertising data records based on a firstplurality of temporal data categories, each temporal data category ofthe first plurality of temporal data categories representing a differentunit of time, and each intermediate data object of the plurality ofintermediate data objects being associated with a temporal data categoryof the first plurality of temporal data categories. The systems may alsoinclude a third processing node configured to generate a plurality ofperformance data objects by partitioning the plurality of intermediatedata objects based on a second plurality of temporal data categories,the second plurality of temporal data categories being different thanthe first plurality of temporal data categories, and each performancedata object of the plurality of performance data objects beingassociated with a temporal data category of the second plurality oftemporal data categories.

In some embodiments, the first processing node is further configured togenerate a plurality of aggregate data objects by partitioning theonline advertising data records based on a third plurality of temporaldata categories, the plurality of aggregate data objects being generatedincrementally, and the generating of the plurality of aggregate dataobjects being responsive to receiving at least some of the plurality ofonline advertising data records. Moreover, the plurality of intermediatedata objects may be generated based, at least in part, on the pluralityof aggregate data objects. In various embodiments, each intermediatedata object of the plurality of intermediate data objects includes asequential representation of at least one online advertising data recordidentified based on a temporal category of the first plurality ofcategories, and the first plurality of temporal data categories includesat least one of a day, week, and month.

In some embodiments, at least one performance data object of theplurality of performance data objects is generated by combining two ormore intermediate data objects, and where the plurality of intermediatedata objects and the plurality of performance data objects are stored ina data storage system configured to process performance data queries.The systems may further include a fourth processing node configured toreceive a query request from an advertisement campaign managementapplication program interface (API), execute the query on the pluralityof intermediate data objects and the plurality of performance dataobjects stored in the data storage system, and generate a result objectthat includes a result of the query. In various embodiments the resultobject includes a combination of at least one intermediate data objectand at least one performance data object. In some embodiments, thefourth processing node is configured to identify the at least oneintermediate data object and at least one performance data objectincluded in the combination based, at least in part, on the first andsecond temporal data categories associated with the at least oneintermediate data object and at least one performance data object, andinclude the at least one intermediate data object and at least oneperformance data object in the result object.

The systems may also include a fifth processing node configured toidentify a plurality of duplicative data events included in theplurality of online advertising data records, and remove the pluralityof duplicative data events from the plurality of online advertising datarecords. In some embodiments, the systems further include a sixthprocessing node configured to identify a plurality of actions based, atleast in part, on the plurality of data events included in the pluralityof online advertising data records, the plurality of actions beingresponsive to a plurality of impressions included in the onlineadvertisement campaign. In some embodiments, the first processing node,the second processing node, and the third processing node are pipelined.In various embodiments, the plurality of intermediate data objects andthe plurality of performance data objects are stored in a distributeddatabase system.

Also disclosed herein are devices that may include a data aggregatorconfigured to receive a plurality of online advertising data records,the plurality of online advertising data records including a pluralityof data events characterizing a plurality of interactions between atleast one user and an online advertisement campaign, and the pluralityof online advertising data records including timestamp datacharacterizing a plurality of creation dates associated with theplurality of data events. The devices may also include an intermediatedata object generator configured to generate a plurality of intermediatedata objects by partitioning at least some of the plurality of onlineadvertising data records based on a first plurality of temporal datacategories, each temporal data category of the first plurality oftemporal data categories representing a different unit of time, and eachintermediate data object of the plurality of intermediate data objectsbeing associated with a temporal data category of the first plurality oftemporal data categories. The devices may further include a performancedata object generator configured to generate a plurality of performancedata objects by partitioning the plurality of intermediate data objectsbased on a second plurality of temporal data categories, the secondplurality of temporal data categories being different than the firstplurality of temporal data categories, and each performance data objectof the plurality of performance data objects being associated with atemporal data category of the second plurality of temporal datacategories.

In some embodiments, the data aggregator is further configured togenerate a plurality of aggregate data objects by partitioning theonline advertising data records based on a third plurality of temporaldata categories, the plurality of aggregate data objects being generatedincrementally, and the generating of the plurality of aggregate dataobjects being responsive to receiving at least some of the plurality ofonline advertising data records. In various embodiments, eachintermediate data object of the plurality of intermediate data objectsincludes a sequential representation of at least one online advertisingdata record identified based on a temporal category of the firstplurality of categories, and the first plurality of temporal datacategories includes at least one of a day, week, and month. In someembodiments, at least one performance data object of the plurality ofperformance data objects is generated by combining two or moreintermediate data objects, and the plurality of intermediate dataobjects and the plurality of performance data objects are stored in adata storage system configured to process performance data queries.

In some embodiments, the devices further include a query node configuredto receive a query request from an advertisement campaign managementapplication program interface (API), execute the query on the pluralityof intermediate data objects and the plurality of performance dataobjects stored in the data storage system, and generate a result objectthat includes a result of the query, where result object includes acombination of at least one intermediate data object and at least oneperformance data object. In some embodiments, the query node may beconfigured to identify the at least one intermediate data object and atleast one performance data object included in the combination based, atleast in part, on the first and second temporal data categoriesassociated with the at least one intermediate data object and at leastone performance data object, and include the at least one intermediatedata object and at least one performance data object in the resultobject.

Also disclosed herein are one or more non-transitory computer readablemedia having instructions stored thereon for performing a method, themethod including receiving a plurality of online advertising datarecords, the plurality of online advertising data records including aplurality of data events characterizing a plurality of interactionsbetween at least one user and an online advertisement campaign, and theplurality of online advertising data records including timestamp datacharacterizing a plurality of creation dates associated with theplurality of data events. The method may further include generating aplurality of intermediate data objects by partitioning at least some ofthe plurality of online advertising data records based on a firstplurality of temporal data categories, each temporal data category ofthe first plurality of temporal data categories representing a differentunit of time, and each intermediate data object of the plurality ofintermediate data objects being associated with a temporal data categoryof the first plurality of temporal data categories. The method may alsoinclude generating a plurality of performance data objects bypartitioning the plurality of intermediate data objects based on asecond plurality of temporal data categories, the second plurality oftemporal data categories being different than the first plurality oftemporal data categories, and each performance data object of theplurality of performance data objects being associated with a temporaldata category of the second plurality of temporal data categories.

In some embodiments, the method further comprises receiving a queryrequest from an advertisement campaign management application programinterface (API), executing the query on the plurality of intermediatedata objects and the plurality of performance data objects, andgenerating a result object that includes a result of the query, whereinresult object includes a combination of at least one intermediate dataobject and at least one performance data object.

Details of one or more embodiments of the subject matter described inthis specification are set forth in the accompanying drawings and thedescription below. Other features, aspects, and advantages will becomeapparent from the description, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of an advertiser hierarchy, implemented inaccordance with some embodiments.

FIG. 2 illustrates a diagram of an example of a system for processingonline advertising data, implemented in accordance with someembodiments.

FIG. 3 illustrates a diagram of an example of a performance dataanalyzer that may be configured to generate one or more data objectsthat facilitate the analysis of online advertising data, implemented inaccordance with some embodiments.

FIG. 4 illustrates a diagram of an example of a system for loading datainto a distributed database, implemented in accordance with someembodiments.

FIG. 5 illustrates a diagram of an example of a system for querying adistributed database, implemented in accordance with some embodiments.

FIG. 6 illustrates a diagram of an example of a query system,implemented in accordance with some embodiments.

FIG. 7 illustrates a flow chart of an example of a data processingmethod, implemented in accordance with some embodiments.

FIG. 8 illustrates a flow chart of an example of another data processingmethod, implemented in accordance with some embodiments.

FIG. 9 illustrates a flow chart of an example of yet another dataprocessing method, implemented in accordance with some embodiments.

FIG. 10 illustrates a data processing system configured in accordancewith some embodiments.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth inorder to provide a thorough understanding of the presented concepts. Thepresented concepts may be practiced without some or all of thesespecific details. In other instances, well known process operations havenot been described in detail so as to not unnecessarily obscure thedescribed concepts. While some concepts will be described in conjunctionwith the specific examples, it will be understood that these examplesare not intended to be limiting.

In online advertising, advertisers often try to provide the best ad fora given user in an online context. Advertisers often set constraintswhich affect the applicability of the advertisements. For example, anadvertiser might try to target only users in a particular geographicalarea or region who may be visiting web pages of particular types for aspecific campaign. Thus, an advertiser may try to configure a campaignto target a particular group of end users, which may be referred toherein as an audience. As used herein, a campaign may be anadvertisement strategy which may be implemented across one or morechannels of communication. Furthermore, the objective of advertisers maybe to receive as many user actions as possible by utilizing differentcampaigns in parallel. As previously discussed, an action may be thepurchase of a product, filling out of a form, signing up for e-mails,and/or some other type of action. In some embodiments, actions or useractions may be advertiser-defined and may include an affirmative actperformed by a user, such as inquiring about or purchasing a productand/or visiting a certain page.

In various embodiments, an ad from an advertiser may be shown to a userwith respect to publisher content, which may be a website or mobileapplication if the value for the ad impression opportunity is highenough to win in a real-time auction. Advertisers may determine a valueassociated with an ad impression opportunity by determining a bid. Insome embodiments, such a value or bid may be determined based on theprobability of receiving an action from a user in a certain onlinecontext multiplied by the cost-per-action goal an advertiser wants toachieve. Once an advertiser, or one or more demand-side platforms thatact on their behalf, wins the auction, it is responsible to pay theamount that is the winning bid.

Data objects and data events associated with advertisement campaignactivity may be generated by various entities, such as servers andbrowsers, during the implementation of an advertisement campaign.Accordingly, such data may be performance data that may be indicative ofa performance of one or more advertisement campaigns. For example, suchdata may be analyzed to determine various performance metrics, such as areturn-on-investment, which may characterize or describe areturn-on-investment provided by an advertisement campaign during aparticular period of time. Because a system may have millions ofassociated users, the amount of performance data generated during a timeperiod being analyzed may be very large. For example, in a single day,many terabytes of records and log files may be generated by a singleadvertisement campaign implemented by an advertiser. The performancedata may include metadata, such as a timestamp, for each record in theraw logs. Based on the timestamp, performance data may be analyzed for adesignated time or data range. For example, an advertiser may requestperformance metrics for an advertisement campaign's performance over thepast thirty days. The performance data may include data objects and dataevents identifying a number of impressions, clicks, actions, revenue,and/or inventory cost for the advertisement campaign.

Conventional techniques for analyzing performance data remain limitedbecause they are not able to effectively and efficiently process queriesfor performance data associated with advertisement campaigns.Conventional techniques may implement burdensome database queries thatare not practical to implement in real time due to the large amount ofperformance data being analyzed, and being constantly generated as theonline advertisement activity continues. Moreover, such queries are notperformed incrementally and result in large and inefficient redundanciesin data processing. Furthermore, conventional techniques are not able tomaintain cached performance data for extended periods of time or withconsistency, thus making the analysis of such data impractical toimplement in a real-time environment. Accordingly, conventionaltechniques are not scalable and cannot be implemented with an efficiencyor consistency that enables efficient real-time analysis of performancedata associated with online advertisement campaigns.

Various methods, systems, and devices are disclosed herein that providea data processing pipeline for processing performance data associatedwith online advertisement campaigns. In various embodiments, the dataprocessing pipeline may be a time series based pipeline that may includeone or more processing stages to pre-process or pre-compute time seriesdata. For example, as disclosed herein, a system may aggregate raw logfiles and data into aggregate data objects to implement an initial stageof processing that arranges the raw data based on a collection orgeneration date. The system may generate several intermediate dataobjects that represent performance data over different designated timeperiods, such as days, weeks, and months. The system may then generateperformance data objects which may represent performance data over aspecific time period which may be determined by an entity, such as anadvertiser, or may be determined based on one or more defaultconfigurations. The performance data objects may be stored or cached ina data storage system. Accordingly, when performance data issubsequently requested by an advertiser for analysis, the generatedperformance data objects may be queried, thus reducing burdensomequeries that might otherwise be performed to obtain the underlying data,and thus enabling the efficient and consistent analysis of performancedata.

FIG. 1 illustrates an example of an advertiser hierarchy, implemented inaccordance with some embodiments. As previously discussed, advertisementservers may be used to implement various advertisement campaigns totarget various users or an audience. In the context of onlineadvertising, an advertiser, such as the advertiser 102, may display orprovide an advertisement to a user via a publisher, which may be a website, a mobile application, or other browser or application capable ofdisplaying online advertisements. The advertiser 102 may attempt toachieve the highest number of user actions for a particular amount ofmoney spent, thus maximizing the return on the amount of money spent.Accordingly, the advertiser 102 may create various different tactics orstrategies to target different users. Such different tactics and/orstrategies may be implemented as different advertisement campaigns, suchas campaign 104, campaign 106, and campaign 108, and/or may beimplemented within the same campaign. Each of the campaigns and theirassociated sub-campaigns may have different targeting rules which may bereferred to herein as an audience segment. For example, a sports goodscompany may decide to set up a campaign, such as campaign 104, to showgolf equipment advertisements to users above a certain age or income,while the advertiser may establish another campaign, such as campaign106, to provide sneaker advertisements towards a wider audience havingno age or income restrictions. Thus, advertisers may have differentcampaigns for different types of products. The campaigns may also bereferred to herein as insertion orders.

Each campaign may include multiple different sub-campaigns to implementdifferent targeting strategies within a single advertisement campaign.In some embodiments, the use of different targeting strategies within acampaign may establish a hierarchy within an advertisement campaign.Thus, each campaign may include sub-campaigns which may be for the sameproduct, but may include different targeting criteria and/or may usedifferent communications or media channels. Some examples of channelsmay be different social networks, streaming video providers, mobileapplications, and web sites. For example, the sub-campaign 110 mayinclude one or more targeting rules that configure or direct thesub-campaign 110 towards an age group of 18-34 year old males that use aparticular social media network, while the sub-campaign 112 may includeone or more targeting rules that configure or direct the sub-campaign112 towards female users of a particular mobile application. Assimilarly stated above, the sub-campaigns may also be referred to hereinas line items.

Accordingly, an advertiser 102 may have multiple different advertisementcampaigns associated with different products. Each of the campaigns mayinclude multiple sub-campaigns or line items that may each havedifferent targeting criteria. Moreover, each campaign may have anassociated budget which is distributed amongst the sub-campaignsincluded within the campaign to provide users or targets with theadvertising content.

FIG. 2 illustrates a diagram of an example of a system for processingonline advertising data, implemented in accordance with someembodiments. As similarly discussed above, in the context of onlineadvertising, millions of users may be presented with advertisementcontent managed by advertisement campaigns that may be implemented viamany channels of communication. In various embodiments, processing suchlarge amounts of data may result in highly accurate performance metricsassociated with the advertisement campaigns. Accordingly, a system, suchas system 200, may be implemented to receive data characterizinginteractions between the users and the advertisement campaigns.Moreover, system 200 may be implemented to process the large amounts ofdata generated by these interactions such that subsequent queries forthe data may be executed quickly, efficiently, and in real-time, thusensuring that an advertiser may quickly and effectively analyze theperformance of an advertisement campaign based on the most recent andup-to-date data.

In various embodiments, system 200 may include one or more presentationservers, such as presentation servers 202. According to someembodiments, presentation servers 202 may be configured to aggregatevarious online advertising data from several data sources. The onlineadvertising data may include live internet data traffic that may beassociated with users, as well as variety of supporting tasks. Forexample, the online advertising data may include one or more data valuesidentifying various impressions, clicks, data collection events, and/orbeacon fires that may characterize interactions between users and one ormore advertisement campaigns. As discussed herein, such data may also bedescribed as performance data that may form the underlying basis ofanalyzing a performance of one or more advertisement campaigns. In someembodiments, presentation servers 202 may be front-end servers that maybe configured to process a large number of real-Internet users, andassociated SSL handling. The front-end servers may be configured togenerate and receive messages to communicate with other servers insystem 200. In some embodiments, the front-end servers may be configureto perform logging of events that are periodically collected and sent toadditional components of system 200 for further processing.

As similarly discussed above, presentation servers 202 may becommunicatively coupled to one or more data sources such as browser 204and server 206. In some embodiments, browser 204 may be an Internetbrowser that may be running on a client machine associated with a user.Thus, a user may use browser 204 to access the Internet and receiveadvertisement content via browser 204. Accordingly, various clicks andother actions may be performed by the user via browser 204. Moreover,browser 204 may be configured to generate various online advertisingdata described above. For example, various cookies, advertisementidentifiers, beacon fires, and user identifiers may be identified bybrowser 204 based on one or more user actions, and may be transmitted topresentation servers 202 for further processing. As discussed above,various additional data sources may also be communicatively coupled withpresentation servers 202 and may also be configured to transmit similaridentifiers and online advertising data based on the implementation ofone or more advertisement campaigns by various advertisement servers,such as advertisement servers 208 discussed in greater detail below. Forexample, the additional data servers may include servers 206 which mayprocess bid requests and generate one or more data events associatedwith providing online advertisement content based on the bid requests.Thus, servers 206 may be configured to generate data eventscharacterizing the processing of bid requests and implementation of anadvertisement campaign. Such bid requests may be transmitted topresentation servers 202.

In various embodiments, system 200 may further include recordsynchronizer 207 which may be configured to receive one or more recordsfrom various data sources that characterize the user actions and dataevents described above. In some embodiments, the records may be logfiles that include one or more data values characterizing the substanceof the user action or data event, such as a click or conversion. Thedata values may also characterize metadata associated with the useraction or data event, such as a timestamp identifying when the useraction or data event took place. According to various embodiments,record synchronizer 207 may be further configured to transfer thereceived records, which may be log files, from various end points, suchas presentation servers 202, browser 204, and servers 206 describedabove, to a data storage system, such as data storage system 210described in greater detail below. Accordingly, record synchronizer 207may be configured to handle the transfer of log files from various endpoints located at different locations throughout the world todistributed file system 210 as well as other components of system 200,such as performance data analyzer 216 discussed in greater detail below.In some embodiments, record synchronizer 207 may be configured andimplemented as a MapReduce system that is configured to implement aMapReduce job to directly communicate with a communications port of eachrespective endpoint and periodically download new log files.

As discussed above, system 200 may further include advertisement servers208 which may be configured to implement one or more advertisementoperations. For example, advertisement servers 208 may be configured tostore budget data associated with one or more advertisement campaigns,and may be further configured to implement the one or more advertisementcampaigns over a designated period of time. In some embodiments, theimplementation of the advertisement campaign may include identifyingactions or communications channels associated with users targeted byadvertisement campaigns, placing bids for impression opportunities, andserving content upon winning a bid. In some embodiments, the content maybe advertisement content, such as an Internet advertisement banner,which may be associated with a particular advertisement campaign. Theterms “advertisement server” and “advertiser” are used herein generallyto describe systems that may include a diverse and complex arrangementof systems and servers that work together to display an advertisement toa user's device. For instance, this system will generally include aplurality of servers and processing nodes for performing differenttasks, such as bid management, bid exchange, advertisement and campaigncreation, content publication, etc. Accordingly, advertisement servers208 may be configured to generate one or more bid requests based onvarious advertisement campaign criteria. As discussed above, such bidrequests may be transmitted to servers 206.

In various embodiments, system 200 may include data storage system 210.In some embodiments, data storage system 210 may be implemented as adistributed file system. As similarly discussed above, in the context ofprocessing online advertising data from the above described datasources, there may be many terabytes of log files generated every day.Accordingly, data storage system 210 may be implemented as a distributedfile system configured to process such large amounts of data. In oneexample, data storage system 210 may be implemented as a Hadoop®Distributed File System (HDFS) that includes several Hadoop® clustersspecifically configured for processing and computation of the receivedlog files. For example, data storage system 210 may include two Hadoop®clusters where a first cluster is a primary cluster including oneprimary namenode, one standby namenode, one secondary namenode, oneJobtracker, and one standby Jobtracker. The second node may be utilizedfor recovery, backup, and time-costing query. In some embodiments, datastorage system 210 may be implemented within the context ofgeographically distributed data centers having about 100% fail-overredundancy, and about 99.99% uptime. Accordingly, data storage system210 may be implemented in one or more data centers utilizing anysuitable multiple redundancy and failover techniques.

In various embodiments, system 200 may also include database system 212which may be configured to store data generated by performance dataanalyzer 216, discussed in greater detail below. In some embodiments,database system 212 may be implemented as one or more clusters havingone or more nodes. For example, database system 212 may be implementedas a four-node RAC cluster. Two nodes may be configured to processsystem metadata, and two nodes may be configured to process variousonline advertisement data, which may be performance data, that may beutilized by performance data analyzer 216. In various embodiments,database system 212 may be implemented as a scalable database systemwhich may be scaled up to accommodate the large quantities of onlineadvertising data handled by system 200. For example, database system 212may be implemented as 40 MySQL instances in a distributed databasesystem. Additional instances may be generated and added to databasesystem 212 by making configuration changes, but no additional codechanges. In various embodiments, database system 212 may be implementedas a combination of RAC clusters and a scalable distributed databasesystem. As will be discussed in greater detail below data storage system210 and database system 212 may be coupled to a query server or querysystem, such as query system 213.

In various embodiments, database system 212 may be communicativelycoupled to console servers 214 which may be configured to execute one ormore front-end applications. For example, console servers 214 may beconfigured to provide application program interface (API) basedconfiguration of advertisements and various other advertisement campaigndata objects. Accordingly, an advertiser may interact with and modifyone or more advertisement campaign data objects via the console servers.In this way, specific configurations of advertisement campaigns may bereceived via console servers 214, stored in database system 212, andaccessed by advertisement servers 208 which may also be communicativelycoupled to database system 212. Moreover, console servers 214 may beconfigured to receive requests for analyses of performance data, and maybe further configured to generate one or more messages that transmitsuch requests to other components of system 200.

In various embodiments, performance data analyzer 216 may be configuredto process performance data included in the online advertisement data toenable efficient and effective updating of the performance data storedin database system 212. For example, performance data analyzer 216 mayinclude one or more data processing nodes, such as deduplicationanalyzer 218. In some embodiments, deduplication analyzer 218 may beconfigured to analyze the received performance data, identifyduplicative data objects, and remove the duplicative data objects. Forexample, performance data associated with a particular advertisementcampaign may include duplicative data objects generated by an end-userclicking on an advertisement from a particular advertisement campaignmultiple times. In this example, the performance data may be processedto identify only the first click. Accordingly, deduplication analyzer218 may be configured to remove or delete all subsequent data objectsidentifying the subsequent clicks. In some embodiments, deduplicationanalyzer 218 may be configured to deduplicate the performance data byimplementing a MapReduce job which, as discussed in greater detail belowwith reference to FIG. 8, may include a mapping phase configured to readlog files and partition them, and a reducing phase configured toidentify and discard duplicative data events, such as clicks. In variousembodiments, deduplication implemented by deduplication analyzer 218removes clicks that may be performed by bots, which may be identifiedbased on a comparison of one or more identifier strings with a list ofrobot strings available from a third party data source, such as theInteractive Advertising Bureau (IAB).

In various embodiments, performance data analyzer 216 may also includedata event analyzer 220. According to various embodiments, data eventanalyzer 220 may be configured to analyze the received performance data,identify one or more data events included in the performance data, anddetermine a relationship between one or more advertisements orimpressions and the one or more actions. Accordingly, as will bediscussed in greater detail below with reference to FIG. 8, data eventanalyzer 220 may be configured to process the received performance datato properly identify particular actions associated with particularadvertisements or impressions to accurately characterize the performanceof one or more advertisement campaigns implemented by system 200. Invarious embodiments, once the actions have been processed by data eventanalyzer 220, a transaction log record may be generated and used byanother system component, such as data analyzer 222 discussed in greaterdetail below.

System 200 may further include data analyzer 222 which may be configuredto aggregate the performance data and populate a data store used tostore and maintain the performance data, such as database system 212. Asdiscussed in greater detail below with reference to FIG. 3, dataanalyzer 222 may include one or more components configured to aggregate,arrange, and cache the performance data to enable storage of the largeamounts of performance data in database system 212. As previouslydiscussed, conventional techniques are not able to support theprocessing of such large amounts of data which may be generated andupdated frequently. As will be discussed in greater detail below, dataanalyzer 222 may be configured to include a data aggregator, anintermediate data object generator, and a performance data objectgenerator which collectively process the received performance data toreduce the processing burden of subsequent queries of the performancedata. In some embodiments, the processed data may be stored in databasesystem 212 and/or data storage system 210. Once stored, the processedperformance data may be queried by an API which may be executed by, forexample, console servers 214.

FIG. 3 illustrates a diagram of an example of a performance dataanalyzer that may be configured to generate one or more data objectsthat facilitate the analysis of online advertising data, implemented inaccordance with some embodiments. As similarly discussed above withreference to FIG. 2, systems as disclosed herein may include one or moreperformance data analyzers, such as performance data analyzer 300, thatmay be configured to process large amounts of performance dataassociated with advertisement campaigns

As similarly discussed above with reference to FIG. 2, performance dataanalyzer 300 may include deduplication analyzer 302 which may beconfigured to analyze the received performance data, identifyduplicative data objects, and remove the duplicative data objects. Aswill be discussed in greater detail below with reference to FIG. 8,deduplication analyzer 302 may be configured to identify and remove suchduplicative data objects or data events thus reducing the overall amountof data processed by performance data analyzer 300. Moreover, aspreviously discussed, performance data analyzer 300 may include dataevent analyzer 304 which may be configured to analyze receivedperformance data, identify one or more data events included in theperformance data, and determine a relationship between one or moreadvertisements or impressions and the one or more actions. As will bediscussed in greater detail below with reference to FIG. 8, data eventanalyzer 304 may be configured to process the received performance datato properly identify particular actions associated with particularadvertisements or impressions to accurately characterize the performanceof one or more advertisement campaigns implemented by an advertiser.

As discussed above with reference to FIG. 2, performance data analyzer300 may further include data analyzer 305 which may further include dataaggregator 306. In some embodiments, data aggregator 306 may beconfigured to aggregate received log files received from various datasources, such as a presentation server. Data aggregator 306 may beconfigured to package the aggregated log files in one or more aggregatedata objects. Accordingly, data aggregator may perform an initial dataprocessing operation on the raw log data that has been received from thevarious data sources. As will be discussed in greater detail below withreference to FIG. 8, data aggregator 306 may be configured to aggregatelog files associated with data events into data objects which may bearranged or partitioned based on a characteristic or feature of the dataevents. Data aggregator 306 may be further configured to store theaggregate data objects in a data storage system.

In various embodiments, data analyzer 305 may further includeintermediate data object generator 308 which may be configured tofurther arrange or package the aggregate data objects into one or moreintermediate data objects. In various embodiments, the intermediate dataobjects may have various designated or predetermined time ranges thatmay be identified based on associated temporal data categories, such asa day, week, month, or year. Thus, intermediate data object generator308 may be configured to perform an additional processing operation onthe performance data that may include the aggregated data objects intolarger data structures, such as the intermediate data objects, based onone or more characteristics or features of the aggregated data objects,such as a timestamp. As will be discussed in greater detail below withreference to FIG. 8, the aggregate data objects may be partitioned orgrouped into various intermediate data objects having varyinggranularities of time ranges.

In various embodiments, data analyzer 305 may also include performancedata object generator 310 which may be configured to generateperformance data objects that identify or include performance data for aparticular time range. Accordingly, performance data object generator310 may be configured to generate a data object that includesperformance data over a particular time range which may have beenpreviously specified by an advertiser or may be specified by a defaultvalue or configuration. Accordingly, a performance data object mayinclude sufficient data to support an analysis of the performance of anadvertisement campaign over a time range that may be requested by anadvertiser. As will be discussed in greater detail below with referenceto FIG. 8, a performance data object may be assembled or generated basedon a combination of several intermediate data objects.

In various embodiments, performance data analyzer 300 or any of itsrespective components may be include one or more processing devicesconfigured to process performance data associated with advertisementcampaigns. In some embodiments, performance data analyzer 300 mayinclude one or more communications interfaces configured tocommunicatively couple performance data analyzer 300 to other componentsand entities, such as a data storage system and a record synchronizer.Furthermore, as similarly stated above, performance data analyzer 300may include one or more processing devices specifically configured toprocess performance data associated with data events and advertisementcampaigns. In one example, performance data analyzer 300 includes atleast one query node and a plurality of big data processing nodes forprocessing large amounts of performance data in a distributed manner.For example, performance data analyzer 300 may include one or more querynodes to handle queries associated with a data storage system, andfurther configured to implement one or more components of a querysystem, as similarly discussed in greater detail below. Moreover, dataanalyzer 300 may further include several processing nodes, configured tohandle processing operations on large data sets. Any suitable number ofnodes may be included in performance data analyzer 300. Accordingly,performance data analyzer 300 may include one or more processing nodesconfigured to implement one or more components of a data aggregator, anintermediate data object generator, and/or a performance data objectgenerator. For example, performance data analyzer 300 may include afirst processing node, a second processing node, a third processingnode, a fourth processing node, a fifth processing node, and/or a sixthprocessing node. In one example, data aggregator 306 may include a bigdata processing nodes for processing large amounts of performance datain a distributed manner. In one specific embodiment, performance dataanalyzer 300 may include one or more application specific processorsimplemented in application specific integrated circuits (ASICs) that maybe specifically configured to process large amounts of data in complexdata sets, as may be found in the context referred to as “big data.”

In some embodiments, the one or more processors may be implemented inone or more reprogrammable logic devices, such as a field-programmablegate array (FPGAs), which may also be similarly configured. According tovarious embodiments, performance data analyzer 300 may include one ormore dedicated processing units that include one or more hardwareaccelerators configured to perform pipelined data processing operations.For example, as discussed in greater detail below, operations associatedwith the generation of intermediate and performance data objects may beprocessed, at least in part, by one or more hardware acceleratorsincluded in intermediate data object generator 308 and performance dataobject generator 310.

In various embodiments, such large data processing contexts may involveperformance data stored across multiple servers implementing one or moreredundancy mechanisms configured to provide fault tolerance for theperformance data. In some embodiments, a MapReduce-based framework ormodel may be implemented to analyze and process the large data setsdisclosed herein. Furthermore, various embodiments disclosed herein mayalso utilize other frameworks, such as .NET or grid computing.

FIG. 4 illustrates a diagram of an example of a system for loading datainto a distributed database, implemented in accordance with someembodiments. In various embodiments, a system, such as system 400, maybe implemented to load data, such as cache data generated by aperformance data object generator, into a data storage system, which maybe a distributed file system. In various embodiments, system 400 may beimplemented as a combination of a distributed database coordinationsystem and a relational database management system.

Accordingly, system 400 may include server 402 which may be configuredto be a job initiator that may submit a computation job to anothersystem component, such as distributed storage node 404. In someembodiments, distributed storage node 404 may be implemented as part ofa Hadoop cluster. In various embodiments, the computation job may be acache computation that may generate one or more performance dataobjects, as discussed in greater detail below with reference to FIG. 8.Once the computation job has finished, another system component, such asdatabase coordination node 406, may receive metadata associated with thecomputation job. In response to the updating of the metadata, anothersystem component, such as database 408 may load the newly computed dataobjects and delete any old ones that have been replaced. In variousembodiments, the data may be partitioned or sharded before being storedin one or more databases, such as database 408. Accordingly, differentshards may be assigned to different instances of databases. In someembodiments, database 408 may include a Java Virtual Machine (JVM) layerthat may be communicatively coupled to database coordination node 406and is notified of any directory changes in database coordination node406. Moreover, database 408 may include a relational database managementsystem (RDBMS) that may be configured to store performance data.

FIG. 5 illustrates a diagram of an example of a system for querying adistributed database, implemented in accordance with some embodiments. Asystem, such as system 500, may be implemented to query data, such ascache data generated by a performance data object generator, from a datastorage system, which may be a distributed file system. As similarlydiscussed above with reference to FIG. 4, system 500 may includedatabase coordination node 502 and database 504. In various embodiments,system 500 may further include a query system, such as query system 506,which may be configured to receive and execute a query of performancedata stored among distributed databases within system 500, such asdatabase 504. In some embodiments, such a query may be generated by anAPI in response to receiving a request from an entity, such as anadvertiser. Accordingly, query system 506 may be configured to fetchmetadata and identify one or more storage locations, such as databasetables, that may be utilized to return a result of the query.

Depending on the query, query system 506 may fetch data from one ormultiple database instances. In various embodiments, query system 506may return the result of the query, or may be configured to merge theresults of the query into a single data object. For example, a query maybe received that includes a request for performance data associated withadvertisements A, B, C for the past week as well as the past Monday.Query system 506 may be configured to determine a number of cached dataobjects, such as performance data objects, that may be retrieved tocomplete the query. In this example, query system 506 may determine thattwo advertiser-specific cached performance data objects should beretrieved, one for the past week's weekly cache and one for Monday'sdaily cache. Query system 506 may analyze a partitioning or shardingstrategy associated with the data objects. In this example, the strategymay indicate that A and C may be stored in shard 3 and B may be storedon shard 1. If shard 3 and shard 1 are implemented on the same databaseinstance, query system 506 may issue one query to query tables for bothshard 1 and shard 3 to return the requested results. If the shards areimplemented on different instances, query system 506 may issue a queryto two different database instances, and the results may be merged priorto being returned as a result of the original query. In someembodiments, query system 506 may be implemented as a query system,discussed in greater detail below with reference to FIG. 6.

FIG. 6 illustrates a diagram of an example of a query system,implemented in accordance with some embodiments. A query system, such asquery system 602, may be configured to, among other things, handle queryrequests which may be made by an advertiser, and retrieve one or moreperformance data objects that may have been previously cached to returna result for a query request. Furthermore, query system 602 may beconfigured to transform or translate queries received from anapplication that may be run by a client machine or console server, suchas console servers 604 and 606, into queries that may be executed upondatabase systems, such as database 610 and database 612.

As similarly discussed above, console servers, such as console servers604 and 606, may execute applications and/or provide onlineadvertisement campaign services, such as those provided by Turn® Inc.,to one or more entities, such as advertisers. For example, an advertisermay manage the implementation of its advertisement campaigns and beprovided with an analysis of associated performance data via one or moreservices provided via an API that may be executed by console servers 604and 606. In some embodiments, requests generated by console servers 604and 606 may be sent as messages having a particular format specific tothe online advertisement campaign service provider associated withconsole servers 604 and 606.

Accordingly, query system 602 may include translator 608 which may beconfigured to transform or translate the server requests into a formatthat may be processed or handled by a database system. For example,translator 608 may translate the requests into SQL statements.Accordingly, the translated requests may be executed by differentdatabases, such as database 610 and database 612. In variousembodiments, the processing of the requests may be performed with arelatively small latency, and a response time perceived at the consoleservers may be a few milliseconds. As will be discussed in greaterdetail below with reference to FIG. 8, in response to receiving arequest for performance data, query system 602 may query time seriesdata included in intermediate data objects as well as cached performancedata objects to identify and retrieve one or more data objects thatsatisfy the query. If a single data object satisfies the query, it maybe retrieved immediately. As discussed in greater detail below, if asingle data object does not satisfy the query, query system 602 may beconfigured to identify several data objects that collectively satisfythe query.

In various embodiments, a query executed by query system 602 may beexecuted across multiple instances of databases in a distributed datastorage system. For example, performance data and associated dataobjects may be stored across numerous different instances of databasesto balance processing loads on the respective databases and ensure loadand access times remain fast. Furthermore, additional processing of thedata may be performed to further decrease response times and decrease anumber of instances that is queried to retrieve data. In someembodiments, data may be replicated and shared among several databasesto make the data available if only one database is queried. For example,query requests may often include a time period or interval that is lessthan three months. Accordingly, if daily performance data is stored inmonthly data tables then up to four different instances may be queriedto retrieve the requested data. If such data is replicated and sharedamong databases, then the number of instances queried may be reduced.

In some embodiments, the replication of data may be determined, at leastin part, based on a replication factor which may identify or indicate anamount of data that should be replicated. Among other things, thereplication factor may be a function of available space in a distributedstorage system. If there is ample space in the storage system and thedata to be replicated, such as performance data objects, is relativelysmall, the data may be replicated across all instances of databaseswithin the storage system. However, if there is not enough space, querysystem 602 may be configured to manage the replication of the data suchthat some free space is left available in the storage system. Forexample, a single copy of data to be replicated may utilize space S_(c).Moreover, a total amount of space available in a distributed storagesystem may be T_(s), a replication factor may be R_(f), and a free spacepercentage limitation may be F_(l). In some embodiments, the free spacepercentage limitation may be about 20%. Accordingly, an replicationfactor appropriate for the free space percentage limitation may bedetermined based on equations 1 and 2 shown below:

$\begin{matrix}{{S_{c} \times {R_{f}/T_{s}}} \leq {1 - F_{l}}} & (1) \\{R_{f} \leq {\left( {1 - F_{l}} \right) \times \frac{T_{s}}{S_{c}}}} & (2)\end{matrix}$

In various embodiments, query system 602 may be further configured todetermine how replicated data is distributed. In one example, a numberof all available instances may be N_(I). Accordingly, if R_(f)=N_(I),then the replication factor may be the number of all availableinstances, and data may be replicated across all instances. However, ifR_(f)=1, then the replication factor may be 1 and data may bedistributed based on available space in the system. Such informationidentifying free space may be maintained by a system component, such adatabase coordination node. In this example, newly generated dataobjects to be replicated may be distributed to the instance that has themost free space. If there are multiple instances that have the sameamount of free space, one may be randomly selected. Moreover, if1<R_(f)<N_(I), then the distribution of data may depend upon a length ofa continuous time interval in a query. For example, a longest continuousinterval covers N_(T) tables. To distribute a data table stored in aparticular instance, metadata managed by the database coordination nodemay be analyzed to determine locations of previous tables within thecontinuous interval. For example, replication factor R_(f) may have avalue of 3, and there may be 40 instances or nodes such that N_(I)=40.As similarly discussed above, query performance may be increased whenmore data objects or tables targeted by the query are stored in fewerdifferent instances because network costs and server loads may bereduced. As discussed above, most queries executed may be within aparticular time or data range. For example, 90% of queries may be fordata within the past three months. As similarly discussed above,performance data may be arranged as time series in intermediate dataobjects. Accordingly, the performance data may be stored in, among othertime periods, one month data objects. Thus, four intermediate dataobjects, and four data tables storing the intermediate data objects, maybe sufficient to satisfy most queries. In this example, of a number ofinstances, R_(f), that include previous data tables within thecontinuous time period, p_(r) percentage may include a copy of a currentdata table. Copies of the current data table may be distributed to theother R_(f) instances based on free space.

For example, if time series data is included in a month-basedintermediate data object, it may be stored in a data object, record, orfile having a naming system that first includes a year and subsequentlyincludes a month based on a creation or collection date, such as 201406.As will be discussed in greater detail below, for other data objectshaving different time-based granularities, additional information, suchas a day, may also be included in the name. It may be determined thatthe data object should be distributed, and metadata may be retrievedfrom a database coordination node to determine the locations of dataobject 201406. A previous data object having a date range for theprevious month, such as data object 201405, may have R_(f) copieslocated in R_(f) instances. Among these R_(f) instances, p_(r)percentage may be stored in the same instance as data object 201406.Accordingly, data object 201406 has been distributed on R_(f)×p_(r)instances, and R_(f)×p_(r) copies have been distributed. The rest of thecopies to be distributed by data replication may be represented byR_(f)−R_(f)×P_(r), and may be distributed based on available free space.Subsequently, a data object for the next month, such as data object201407 may be distributed. Accordingly, p_(r) percentage of data object201406 copies may be stored in the same instance as data object 201407.Thus, R_(f)×p_(r)×p_(r)=R_(f)×p_(r) ², and the instance may store dataobjects 201405, 201406, and 201407. Further still, another data object,such as 201408, may be distributed. As discussed above, p_(r) percentageof data object 201407 copies may be stored in the same instance as dataobject 201408. Accordingly, R_(f)×p_(r) ³ instances include data objects201405, 201406, 201407, and 201408. As discussed above, for an intervalof three months, at most four data tables may be queried. Accordingly,an expectation that an instance includes all data tables may bedetermined based on equations 3 and 4 shown below:

$\begin{matrix}{{R_{f} \times p_{r}^{N_{t} - 1}} > 1} & (3) \\{p_{r} = \sqrt[{N_{t} - 1}]{\frac{1}{R_{f}}}} & (4)\end{matrix}$

If R_(f)×p_(r) ³>1, at most one instance includes data objects 201405,201406, 201407, and 201408. Accordingly, that one instance may beidentified based on available metadata and queried. Similar to anexample above, if R_(f)=3, then p_(r) may be 69.3%.

In some embodiments, query system 602 or another system component may beconfigured to migrate data to consolidate the storage of time seriesdata that may be stored in intermediate data objects as well asperformance data objects that may be cached. As similarly discussedbelow, a unit of time series data may be daily, weekly, and/or monthly.In some embodiments, queries for performance data may not exactlycoincide with the units of the time series. For example, a query fordata collected within the past month may be retrieved from a monthlyperformance data object or intermediate data object. Accordingly, suchdata may be stored in a single data table. However, if queries are fortwo months, the queries may access two different tables. Becauseadditional database table accesses may reduce performance, the data maybe processed to reorganize the data and reduce database table accesses.

In one example, a database table may store one month of performancedata. In this example, queries may be received from an entity, such asan advertiser, for 3 months of data and may call for accesses to 3tables. More specifically, if data objects storing data for differentmonths are stored in different tables, are labeled 201405, 201406, and201407, and are stored in a single instance, a system component, such asquery system 602, may be configured to combine or merge the data objectstogether to generate a larger table which may be labeled 20140567. Thenewly generated table may include all the data for 201405, 201406, and201407. Because the data is now stored in a single table, the databaseperformance may be improved. If 201405, 201406, and 201407 are locatedin two different instances, such as 201405 and 201406 being stored in afirst instance and 201407 being stored in a second instance, 201407 maybe migrated to the first instance, and the data may be combined. Invarious embodiments, the reorganization of data may be performed, atleast in part, dynamically. Thus, a time period over which data objectsshould be combined may be determined dynamically, and based onadvertiser activity. Returning to a previous example, the time period of3 months that included 201405, 201406, and 201407 may have beendetermined based on a large frequency of 3 month requests issued by anadvertiser.

In various embodiments, query system 602 may be configured to identifyand select combinations of data objects which may be stored in the sameinstance of a database. Preferably selecting combinations stored in thesame instance may minimize queries to multiple instances and reduceprocessing times. For example, once query system 602 has identified acombination of data objects that satisfies a query, query system 602 maydetermine whether or not the combination is located in a singleinstance. If the combination is located in the same instance, thatcombination may be selected. If the combination is not located in thesame instance, query system 602 may identify a combination of dataobjects that is stored across the fewest instances, and may select thatcombination, and query system 602 may retrieve the data objects from thedifferent instances. In this way data may be retrieved from multipletables within a database, and from multiple instances of the databasesthemselves.

FIG. 7 illustrates a flow chart of an example of a data processingmethod, implemented in accordance with some embodiments. A dataprocessing method, such as method 700, may be used to implement atime-series based pipelined processing of performance data associatedwith an advertisement campaign. As discussed above, performance datathat has been stored and logged may occupy numerous terabytes of datawhich may be too great in quantity for conventional analysis techniques.Accordingly, a pipelined method may be implemented that may includeseveral stages or layers in a data processing scheme. For example, anaggregation and/or time series stage may be implemented to perform aninitial stage of processing on raw online advertisement data. The resultmay a series of intermediate data objects which may include sequentialrepresentations of the underlying data organized based on several timecategories. An additional stage of processing may be implemented togenerate performance data objects organized based on different timecategories. The data objects may be stored and cached for subsequentanalysis. As discussed in greater detail below, the time seriespipelined methods disclosed herein enable the efficient generation oftime series data, and the efficient querying of such data when ananalysis of the underlying performance data is subsequently performed.

Accordingly, method 700 may commence with operation 702 during whichonline advertising data records may be received. In some embodiments,the online advertising data records include records and log files thatrepresent data events related to an online advertisement campaign.Accordingly, the data events may characterize interactions, such asactions, clicks, and views, between a user and an online advertisementcampaign. The data events may have associated metadata, such astimestamps, that identifies when the data was generated, created, and/orcollected.

Method 700 may proceed to operation 704 during which the received onlineadvertising data may be arranged or packaged into several other dataobjects. As similarly discussed above, operation 704 may be part of atime series based pipelined process. Thus, during operation 704, aninitial stage of data processing may be implemented and severalintermediate data objects may be generated by partitioning at least someof the online advertising data records into different data objects basedon a first set of temporal data categories. In some embodiments,temporal data categories may be data categories that represent units oftime, such as days, weeks, months, and years. Accordingly, the onlineadvertising data may be aggregated into different data objects togenerate or pre-compute several sequential representations or timeseries of the online advertisement data. As discussed above, theintermediate data objects may have different temporal data categories.Thus, different intermediate data objects may have different associatedunits of time and may represent different durations of time. In thisway, the raw online advertising data may be packaged into sequentialdata objects of varying durations or lengths, as may be determined basedon one or more designated or predetermined time periods or intervals. Aswill be discussed in greater detail below with reference to FIG. 8, theraw online advertising data may be processed or aggregated intoaggregate data objects, and the intermediate data objects may begenerated incrementally based on the aggregate data objects.

Method 700 may proceed to operation 706 during which several performancedata objects may be generated by partitioning the intermediate dataobjects into different data objects based on a second set of temporaldata categories. Accordingly, operation 706 may implement an additionalstage of processing which further combines the online advertising datainto additional sequential data objects that have a different set oftemporal data categories than those discussed above with reference tothe intermediate data objects. Accordingly, as will be discussed ingreater detail below with reference to FIG. 8, the performance dataobjects may have a larger temporal duration than the intermediate dataobjects, and may have a duration or date range that is customized ortargeted to an advertisement campaign and an analysis of the performanceof the advertisement campaign. As discussed below, such performance dataobjects may be stored in a data storage system and cached such that theperformance data objects are rapidly accessible when queried, andadditional processing of the underlying data is not needed.

FIG. 8 illustrates a flow chart of an example of another data processingmethod, implemented in accordance with some embodiments. As similarlydiscussed above, performance data may be processed to enable the quickand efficient execution of subsequent queries on such performance data.In various embodiments, the processing may include a time series basedpipelined method, such as method 800, in which system components mayprocess the data in different layers or stages to pre-compute one ormore data objects that are arranged to represent time seriesinformation, and are stored and cached to be quickly accessible when aquery is executed. As will be discussed in greater detail below, thetime series based pipelined method may be an incremental method, thusreducing redundancies in an overall processing overhead which may beincurred.

Accordingly, method 800 may commence with operation 802 during whichperformance data associated with one or more advertisement campaigns maybe received. As similarly discussed above, the performance data mayinclude one or more data values identifying data events, which may beactions associated with advertisement campaigns. The performance datamay be received from various data sources, such as browsers and serversused to implement the advertisement campaigns. The performance data maybe received at one or more presentation servers and may be handled by arecord synchronizer.

Method 800 may proceed to operation 804 during which duplicative dataevents may be identified and removed from the performance data. Assimilarly discussed above, a system component, such as a deduplicationanalyzer, may be configured to analyze the received performance data,identify duplicative data events or objects, and remove the duplicativedata events. In some embodiments, the duplicative data events may beincluded in data objects such as log files which may be generated basedon a user performing various actions, such as clicking on anadvertisement, in response to being presented with that particularadvertisement, as may occur during a decision making process that mayinclude shopping and online commercial transactions of the user. Thus,multiple data events and associated log files may be included in thereceived performance data. The log files may include data values thatidentify the user, identify a type of data event, and further identifythe advertisement associated with the data event. In variousembodiments, the deduplication analyzer may identify the log filesassociated with the user and advertisement, and the deduplicationanalyzer may discard all duplicative data events to reduce the amount ofdata that is subsequently processed.

For example, the deduplication analyzer may implement a MapReduce jobwhich may first read the received performance data. The performance datamay include new log files identifying new data events, such as clicks.The received performance data may also include older data that may havebeen previously processed and deduplicated during a previous iterationof method 800. The deduplication analyzer may generate a plurality ofadvertisement-specific data objects by partitioning log files thatidentify a particular data event, such as a click, based on anadvertisement identifier included in the log files. The deduplicationanalyzer may subsequently reduce or deduplicate theadvertisement-specific data objects based on timestamp metadataassociated with the log files as well as status data associated with thelog files. In some embodiments, the status data may be one or more datavalues that include a flag or identifier that indicates whether or not alog file has been previously processed by method 800. For example, ifall of the data events included in a particular advertisement-specificdata object have been received from log files that are new and have notbeen previously processed, then the data event included in the log filethat has the earliest timestamp may be retained, and the other dataevents may be discarded. In this way, the data included in theadvertisement-specific data object may be reduced or deduplicated to asingle data event. Furthermore, if the first data event in anadvertisement-specific data object was included in a log file that hasalready been processed by method 800, then all subsequent data eventsmay be discarded.

In various embodiments, deduplication may be optionally performed bydeduplication analyzer, or may be performed by another component, suchas a data aggregator. For example, all received performance data may beprovided to the data aggregator and the data aggregator may filter theperformance data based on an implementation of a MapReduce job.

Method 800 may proceed to operation 806 during which one or more actionsincluded in the performance data may be identified. As similarlydiscussed above, a system component, such as a data event analyzer, maybe configured to analyze the received performance data, and identify oneor more actions included in the performance data based on a relationshipbetween one or more data events and advertisements or impressions. Aspreviously discussed, an action may be an event that identifies anactivity or operation performed by an end user. For example, an end usermay perform an action by clicking on an advertisement, viewing anadvertisement, or performing a conversion, as may occur when purchasinga product. In this example, the end user may arrive at a webpageassociated with the conversion at the conclusion of making a purchase.In some embodiments, the webpage may include a tracking pixel that maybe configured to send a message to one or more servers, such as anadvertisement server and/or presentation server, identifying the loadingof the page and the purchase of the product. In response to receivingthe message, the presentation server may generate a log file thatidentifies the data event that just occurred, which may be a beaconevent.

In some embodiments, an action may be identified based on the data eventby correlating the data event to an impression that occurred prior tothe data event. For example, if the beacon event may be correlated withan advertisement that was previously presented to the end user, the dataevent may be identified as an action or the advertisement may beidentified as a type of action, depending on one or more advertiserpreferences. In various embodiments, a relationship between a data eventand an impression may be determined based on one or more user historyrecords that may be stored for each user identified by the system. Auser history record may store a list or log of all impressions and dataevents associated with a particular user. Such a record may be storedand maintained for each of millions of users associated with the systemused to implement method 800.

Accordingly, the log file associated with the data event being analyzedmay include a user identifier that identifies the end user who performedthe activity that generated the data event. The user identifier may beused to identify a particular user history record for that end user. Invarious embodiments, new log files that have not been previouslyprocessed may be partitioned based on user identifier. Each partitionfor a user may be merged with an existing user history record associatedwith that user. According to some embodiments, the contents of themerged list now included in the user history record may be analyzed toidentify impressions that have been served to the user. In someembodiments, the clicks and impressions included in the user historyrecord may be filtered based on an identifier associated with the dataevent. For example, a data event, such as a beacon event or fire, mayinclude an identifier which may have been determined by an advertiserand associated with a particular advertisement campaign. Several clicksand/or impressions included in the user history record may be identifiedbased on the identifier. In some embodiments, the most recent click ormost recent impression may be identified as a click view based action ora view based action.

In various embodiments, operation 806 may be performed subsequent tooperation 804. In some embodiments, operation 806 may be performed intandem with or prior to operation 804. Accordingly, operation 804 andoperation 806 may be implemented independently of each other, andoperation 804 and operation 806 may be executed concurrently uponreceiving the performance data at operation 802.

Method 800 may proceed to operation 808 during which log files that havebeen received may be aggregated into a plurality of aggregate dataobjects. In some embodiments, the log files aggregated during operation808 may have been previously processed in accordance with operations 804and 806. Accordingly, previously received performance data may have beenpre-processed to eliminate duplicative data events and accuratelyidentify actions associated with the data events. In variousembodiments, the log files associated with the data events may beaggregated into data objects which may be arranged or partitioned basedon a characteristic or feature of the data events. Accordingly, the logfiles may be partitioned into aggregate data objects based on timestampsassociated with the log files. For example, a MapReduce job may beimplemented to generate aggregate data objects partitioned based on acollection day. In this example, the log files collected or received ona first date, such as 11/20/14, may be stored in a first aggregate dataobject, and log files collected or received on a second date, such as11/21/14 may be stored in a second aggregated data object. As will bediscussed in greater detail below, aggregate data objects may begenerated by numerous iterations of method 800 which may occur manytimes over the course of a period of time, such as a day. Thus, numerousaggregate data objects may be stored in a folder that represents thatperiod of time, which may be a day. Accordingly, each aggregate dataobject may include a list of data events and/or log files that werereceived or generated during the designated period of time associatedwith that aggregate data object, which may be a unit of time such asday. In various embodiments, the aggregate data objects may includeidentifiers as well at least some of the performance data. For example,an aggregate data object may include one or more identifiers thatidentify a particular item, such as an advertisement, in anadvertisement campaign or sub-campaign. The aggregate data object mayalso include a collection or count measure which identifies a totalnumber of clicks and/or impressions that occurred during a time periodassociated with the aggregate data object.

As discussed above, the aggregate data objects may be generatedincrementally in successive iterations of method 800. For example, afirst iteration may generate a first aggregate data object that includesor identifies a first set of data events associated with anadvertisement campaign. A successive iteration may generate a secondaggregate data object that may include or identify any data events thatoccurred after the generation of the first aggregate data object. Suchdata events may be identified based on queries of the received raw logfiles. In one example, both the first and second aggregate data objectsmay have been generated on the same day and may be stored in a folderthat represents that day, and may be labeled accordingly with a foldername such as 20140301. In this example, the folder 20140301 may storeseveral aggregate data objects that were generated by differentiterations of method 800 that occurred on 3/1/2014, where each aggregatedata object represents relevant data events that occurred since aprevious iteration of method 800.

As discussed above, the aggregate data objects may be stored in a folderof a data storage system and/or database system. Thus, a data folder maybe generated that includes aggregate data objects generated for logfiles associated with a designated period of time. For example, a datafolder may be generated for a date of 10/10/14 and may include allaggregate data objects that have been generated for log files collectedduring that day. In various embodiments, the aggregation of the data maybe performed at any date relative to the collection dates of the logfiles. For example, log files received on 11/21/14 may be aggregated andstored in an aggregate data object for that particular date during anaggregation operation that has been implemented on a subsequent date,such as 11/24/14. While various examples above have been described withreference to a unit of time that is a day, any suitable unit of time maybe used, such as a week or month. Moreover, aggregate data objects maybe generated in batches. For example, aggregate data object may begenerated for each day over a designated period of time, which may bethe previous three months. In this way, the performance data may beprocessed to generate several aggregate data objects.

Method 800 may proceed to operation 810 during which at least some ofthe performance data may be included in a plurality of intermediate dataobjects. In various embodiments, the aggregate data objects generatedduring operation 808 may be further processed to generate one or moreintermediate data objects. As previously discussed, the aggregate dataobjects may be generated by aggregating raw log files into data objectsbased on a particular characteristic or feature, such as collection orgeneration date. In various embodiments, such an aggregation operationmay be performed numerous times within a designated time period.Accordingly, a folder associated with a particular date, such as10/15/14, may store several different aggregate data objects generatedby different iterations of aggregation operations.

In various embodiments, a system component, such as an intermediate dataobject analyzer, may be configured to generate one or more intermediatedata objects by merging one or more aggregate data objects. In someembodiments, the intermediate data objects may be sequential dataobjects that are configured to store the previously processedperformance data in data objects that provide different sequential ortemporal views of the performance data. As will be discussed in greaterdetail below, aggregate data objects may be merged into intermediatedata objects having different features or characteristics. For example,intermediate data objects may be generated that include log files or oneor more data values characterizing the log files that are organized orarranged by day, week, and/or month. Thus, the incrementalrepresentation of data events stored in the aggregate data objects maybe merged into different intermediate data objects associated withdifferent temporal data categories. It will be appreciated that theintermediate data objects may be generated based on any suitable unit oftime. The intermediate data objects may be stored in a data storagesystem and/or database system. As similarly discussed above, theintermediate data objects may be stored in separate folders organizedaccording to a unit of time. In this way, log files and associated dataevents may be iteratively processed and arranged into separate dataobjects based on one or more characteristics, such as a date ofcollection or generation. Moreover, as described herein, such arrangingof the log files may be performed efficiently in a way that reducesredundancies in processing operations.

According to some embodiments, the intermediate data objects may begenerated incrementally. Thus, aggregate data objects and intermediatedata objects generated based on previous iterations of method 800 mayprovide a basis for the generation of new or updated intermediate dataobjects. A system component, such as an intermediate data objectgenerator, may be configured to scan previously stored intermediate dataobjects and to identify the most recently generated intermediate dataobject. In some embodiments, the intermediate data objects may be storedin a particular format. For example, the intermediate data objects maybe stored in a folder that has the following format as a folder name:<time series type>/<collection date>/<time series type>_<collectiondate>₁₃ <creation date>.<iteration number>. The contents of the foldermay be analyzed to identify the intermediate data object having the mostrecent creation date. Once identified, one or more aggregate dataobjects may be identified based on the creation date. For example,aggregate data objects having a creation dates that occurred later thanthe creation date of the intermediate data object may be identified. Ifthere are multiple aggregate data objects with the same creation date,the aggregate data object having the largest or most recent iterationnumber may be selected. In this way, the most recent aggregate dataobject may be selected and combined with a previously generatedintermediate data object to incrementally update the intermediate dataobjects.

For example, an intermediate data object may have been generated for aparticular day and may be stored in a folder having a pathname of:

Daily/140301/daily_140301_140531.385. In this example, a dailyintermediate data object named “daily_140301_140531.385” may includeaggregate data objects up until the creation of an aggregate data objectnamed 140531 having a particular date represented by 140301. Morespecifically, the aggregate data object may include data from log files140301/mr_140301.105, . . . 140301/mr_140531.385. If a new aggregatedata object is subsequently generated, for example, in another iterationof method 800, a new file 140301/mr_140602.401 may be generated andassociated with 140301. In this example, the new file may be retrievedas new incremental data. In some embodiments, if there are multipleaggregate data objects that have the same creation date, such asaggregate data objects named 140301/mr_140602.398 and140301/mr_140602.401, the aggregate data object having the most recentiteration of method 800 may be selected. In this example, the aggregatedata object 140301/mr_140602.401 may be selected because iterationnumber 401 is larger and more recent than 398. Thus, the new data of140301 may be included with the previously generated data included indaily 140301_140531.385. In this way, the previously generatedintermediate data object and its underlying data do not have to berecomputed, and processing operations may be reduced.

In various embodiments, intermediate data objects may be larger thanaggregate data objects. For example, a size of an intermediate dataobject “daily_140301_140531.385” may be significantly larger than anaggregate data object “140301/mr_140602.401.” Accordingly, aggregatedata objects may be integrated or merged with intermediate data objectsin batches. In some embodiments, a threshold value may determine when abatch merge should occur. For example, a batch merge may be initiated inresponse to a threshold value being met. In this example, the thresholdvalue may be a designated number of aggregate data objects. For example,a batch merge may be initiated when more than 10 new aggregate dataobjects have been identified in accordance with the incremental processdescribed above.

In some embodiments, a batch merge may be initiated in accordance withthe following equations. For example, as shown in equation 5 below:

Cost (k)=Merge_(cost)+Query_(cost)+LeftOver_(cost)   (5)

In this example, Cost may be an estimated cost score associated with avariable k. Merge_(cost) may be a cost score if a merge has beentriggered. Query_(cost) may be a cost score associated with accessingmerged intermediate data objects. LeftOver_(cost) may be a cost scoreassociated with accessing aggregate data objects. Accordingly, equation5 may be implemented to provide an approximation of an overall costincurred. As discussed in greater detail below, such an expression ofcost may be used to estimate the merging boundary k. As previouslydiscussed, the aggregate data object having the most recent creation orcollection data may be selected. Accordingly, for the past D dates ofdata, D/k merges may be initiated. In some embodiments, a weight w_(m)may be generated and assigned to each merge. Accordingly, an estimateMerge_(cost) may be determined by equation 6 shown below:

Merge_(cost) =w _(m) D/k   (6)

In some embodiments, the most recent intermediate data objects may beaccessed more often or frequently than older ones. For example, afrequency of accesses may be linearly and inversely proportional to anamount of time from the present execution of method 800. Accordingly, aweight w_(q) may be generated based on such a relationship. Thus, anaccess cost incurred when accessing a current intermediate data objectmay be w_(q)D, whereas an access cost incurred when accessing anintermediate data object generated previously may be w_(q)(D−1), and forD days ago may be w_(q). Accordingly, a total access cost may bedetermined based on equation 7 shown below:

Query_(cost) =w _(q) D+w _(q)(D−1)+ . . . w _(q) =w _(q)(D+1)D/2   (7)

In some embodiments, there may be one or more aggregate data objectsthat have not yet been merged. In various embodiments, an expect numberof leftover aggregate data objects may be determined based on equation 8shown below:

$\begin{matrix}{E_{num} = \frac{\left( {k - 1} \right)}{2}} & (8)\end{matrix}$

As similarly discussed above, an access frequency associated with theseaggregate data objects may be inversely and linearly proportional to anamount of time from the current execution of method 800. Accordingly, aweight w₀ may be generated for aggregate data objects, and a cost foraccessing aggregate data objects may be determined based on equation 9and equation 10 shown below:

$\begin{matrix}{{LeftOver}_{cost} = {{{w_{o}\left( {{w_{q}\left( {D + 1} \right)}{D/2}} \right)}E_{num}} = {w_{o}{w_{q}\left( {D + 1} \right)}{{D\left( {k - 1} \right)}/4}}}} & (9) \\{\mspace{20mu} {{{Min}\; {Cost}} = {\frac{w_{m}D}{k} + \frac{{w_{q}\left( {D + 1} \right)}D}{2} + \frac{w_{o}{w_{q}\left( {D + 1} \right)}{D\left( {k - 1} \right)}}{4}}}} & (10)\end{matrix}$

In various embodiments, an approximation of cost may be determined basedon equation 11 shown below:

$\begin{matrix}{\frac{\left( {{Min}\; {Cost}} \right)}{k} = 0} & (11)\end{matrix}$

Accordingly, the variable k which may represent a threshold number ofnew aggregate data objects that should be present to trigger a batchmerger may be determined based on equation 12 shown below:

$\begin{matrix}{k = {2\sqrt{\frac{w_{m}}{w_{0}{w_{q}\left( {D + 1} \right)}}}}} & (12)\end{matrix}$

As shown in equation 12, if the merge weight w_(m) is high, k may beincreased to reduce a total cost incurred by the batch merge operation.If the accessing cost weight is high, k may be decreased to trigger abatch merge more frequently, and to decrease or reduce the total costincurred by the batch merge operation. As similarly discussed above,weights may be determined based on one or more usage statisticsassociated with the system. For example, data objects having largersizes may be assigned a higher weight, such as a relatively large w_(q).Moreover, data objects having date ranges that are accessed frequentlymay be assigned higher weights, such as a relatively large w_(m).Similarly, data objects having smaller sizes or date ranges that areaccessed less frequently may be assigned lower weights such as arelatively small w_(q) and w_(m) respectively.

In some embodiments, the generation of the intermediate data objects maybe configured based on one or more entity-specific parameters. Invarious embodiments, an entity-specific parameter may be anadvertiser-specific parameter. For example, the advertiser-specificparameter may be a time zone associated with the advertiser. In thisexample, an advertiser may be located in a particular time zone and mayaccess a database system via a console server also located in that timezone. As described above, performance data may be received andaggregated from multiple different time zones across the world.

For example, performance data may be received from the Pacific time zoneas well as the Eastern time zone. As will be described in greater detailbelow, the intermediate data objects may be generated based, at least inpart, on the advertiser time zone and may be configured specifically forthe advertiser time zone. In particular embodiments, the entity-specificparameter may be a user-specific parameter. Accordingly, theuser-specific parameter may be a time zone associated with a user whomay be an online customer, and the intermediate data objects may begenerated based, at least in part, on the user time zone and may beconfigured specifically for the user time zone.

In various embodiments, a default time zone may be selected. Forexample, a default time zone may be determined to be the Eastern timezone of the United States. In this example, all data aggregated andstored during operation 808 may be converted to the default time zone.Intermediate data objects generated based on the aggregated data mayalso be generated in accordance with the default time zone. Theintermediate data objects may be converted to a target time zone uponstorage in a data storage system and/or database system. For example,the target time zone may be a time zone associated with a user oradvertiser. Alternatively, the intermediate data objects may begenerated in accordance with the target time zone and subsequentlystored with no additional conversion.

In various embodiments, the conversion of log files to the default timezone may be performed by analyzing one or more data values included inthe log file that identify the timestamp and a native time zoneassociated with the log file. Moreover, additional performance data maybe identified based on the native time zone and time stamp of aparticular log file. For example, if performance data is retrieved froma first time zone, a corresponding date in a potential target time zonewill be within one day before or after the timestamp of the log file.Accordingly, to compute daily data D for a target time zone, performancedata from D−1, D, and D+1 as identified by the default time zone may beretrieved. An aggregation operation as described above with reference tooperation 808 may be performed in response to receiving one or more datavalues identifying a particular date in a target time zone.

In some embodiments, the aggregate data objects may be configured toinclude at least two time zone data fields. A first time zone data fieldmay be configured to store a default time zone key and a second datafield may be configured to store a target time zone key. Accordingly, inresponse to identifying a customer time zone, target time zone keys/datamay be generated based on the native or default time zone and time stampof the aggregate data object and log files included in the aggregatedata object. Thus, in response to receiving a request for performancedata from a customer time zone, performance data identified by thestandard time zone may be queried, target time zone keys may becalculated, and intermediate data objects customized for the target timezone may be generated. For example, to retrieve performance data for adate identified by 140302 in a customer time zone, performance data maybe retrieved for 140301, 140302 and 140303 from a standard time zonefolder, such as Daily/140301, Daily/140302, and Daily/140303. Based onthe customer time zone, target time zone keys may be generated and usedto generate an intermediate data object based on the retrievedperformance data. Accordingly, the intermediate data object may becustomized or targeted to the customer's time zone and may be labeledaccordingly by having a file name such as CustomerDaily/140302.

Method 800 may proceed to operation 812 during which one or moreperformance data objects may be generated based on at least some of theintermediate data objects. In various embodiments, the performance dataobjects may include performance data for one or more predetermined ordesignated time or date ranges. As previously discussed, performancedata may have been previously arranged into units of time of varyinggranularities. For example, the performance data may have beenpartitioned into intermediate data objects having associated timeperiods of days, weeks, or months. As will be discussed in greaterdetail below with reference to FIG. 9, performance data objects may begenerated that have varying time or date granularities based on thepreviously generated intermediate data objects. For example, based onthe previously generated intermediate data objects, a system component,such as a performance data object generator, may generate a performancedata object that includes data events identified by a combination ofintermediate data objects. In various embodiments, the intermediate dataobjects may be determined or identified such that the number ofintermediate data objects included in the performance data object isreduced, and an amount of processing operations is also reduced.

Method 800 may proceed to operation 814 during which it may bedetermined whether or not additional performance data has been receivedand should be processed. Such a determination may be made based on oneor more data values received from a system component, such as apresentation server. For example, a system component, such as aperformance data analyzer, may be configured to detect the receiving ofadditional log files from one or more presentation servers, and may beconfigured to set or modify a flag or identifier in response to thedetecting. One or more system components may initiate another iterationof method 800 based on the status of the flag or identifier. If it isdetermined that additional performance data has been received and shouldbe processed, method 800 may return to operation 802. If it isdetermined that no additional data should be processed, method 800 mayterminate.

FIG. 9 illustrates a flow chart of an example of yet another dataprocessing method, implemented in accordance with some embodiments. Assimilarly discussed above, intermediate data objects may be sequentialrepresentations of performance data that has been partitioned ororganized into discrete units of time, such as a day, week, month, andyear. In various embodiments, the intermediate data objects may befurther processed to generate performance data objects. In someembodiments, performance data objects may also be sequentialrepresentations of performance data. However, performance data objectsmay be generated for less uniform or custom periods of time. Forexample, a performance data object may be generated for a fraction of aweek, a fraction of a month, a period of 90 days in the past, or aperiod of 16 months in the past. In such examples, such a date or timeperiod may be determined based on an input that may have been previouslyreceived from an advertiser. Thus, one or more performance data objectsmay be generated and stored as cache data that may be quickly retrievedand utilized when requested by an advertiser.

Accordingly, method 900 may commence with operation 902 during which adesignated time parameter may be determined. As similarly discussedabove, a performance data object may have an associated time period orduration that defines its respective data range. Accordingly, the timeperiod may be defined or characterized by a time parameter. In someembodiments, the time parameter may be a default value or may have beenpreviously received from an entity, such as an advertiser. For example,an advertiser may provide or may have previously provided an input to aconsole server when initiating an analysis of performance data for anadvertisement campaign that has been implemented by the advertiser. Theinput may specify a particular time frame that the advertiser intends toanalyze. Accordingly, such a time frame may provide the basis of thegeneration of a performance data object. In various embodiments, thetime frame may be one of several default or designated settings thatautomatically generates and provides a default set of performance dataobjects to the advertiser. As similarly discussed above, one or moreendpoints of the time frame may be determined based on date and time ofgeneration of the data object. For example, a data object having a daterange of 16 months may have a first end point determined based on thegeneration date of the data object, and may also have a second end pointdetermined by subtracting 16 months from the generation date.

Method 900 may proceed to operation 904 during which a first performancedata object may be generated based on the determined time parameter. Assimilarly discussed above, a performance data object may be generated orpopulated based on a combination of previously generated intermediatedata objects. In this way, a performance data object may be assembledfrom a combination of several intermediate data objects that haveassociated time periods smaller than the determined time parameter. Thecombination of intermediate data objects may be configured to reduce theoverall number of records accessed, and reduce an overall processingtime associated with method 900. In various embodiments, suchcombinations may be determined based on one or more shortest pathtechniques or breadth first searching techniques.

Accordingly, several candidate combinations of intermediate data objectsmay be analyzed to identify a particular combination that may be used togenerate the performance data object. In some embodiments, all availableintermediate data objects having data, such as log files, includedwithin the time range specified by the time parameter may be retrievedfor analysis. The retrieved intermediate data objects may be analyzedsequentially and based on their recency and the relative size of theirtime ranges. In one example, the time parameter may specify a targetendpoint for a time period or range, such as 55 days in the past. Asystem component, such as a performance data object generator, may beconfigured to iteratively analyze and track endpoints of the time rangesassociated with the retrieved intermediate data objects to assemble acollection of intermediate data objects that collectively includes adate or time range that reaches the target endpoint. In this example, alargest available intermediate data object may be selected and itsendpoint may be used as the starting point of the next iteration of theassembly process. For example, the intermediate data object that wasselected may have been for a month and may reach 30 days back. The nextintermediate data object may be selected subsequently. Because a monthis too large and would exceed the 55 day limit, an intermediate dataobject that has a time range of a week may be selected, thus starting atthe 30 day ending point of the previous intermediate data object, andreaching 37 days back. Additional iterations may be performed two moretimes with two additional week-long intermediate data objects to reach51 days back. At this point, a week may be too large, so an intermediatedata object having a time range of one day may be selected, thusreaching back 52 days. The process may be repeated 3 more times toachieve a collection of performance data that spans the entire timeperiod designated by the time parameter, which is 55 days in the past inthis example. Accordingly, the intermediate data objects included in theperformance data object would include 1 month-long intermediate dataobject, 3 week-long intermediate data objects, and 4 day-longintermediate data objects. In various embodiments, such an iterativeassembly process may be implemented in a single instance or node of adata storage or database system. In some embodiments, if additional dataobjects are needed to satisfy the query and cannot be retrieved from asingle instance, additional instances may be queried, as discussed ingreater detail below.

Method 900 may proceed to operation 906 during which the firstperformance data object may be stored. As similarly discussed above, thefirst performance data object may be stored in a data storage systemand/or database system. The first performance data object may besubsequently accessed by an entity, such as an advertiser, whenperformance data is queried by the advertiser. Thus, performance dataobjects may be computed, updated, and cached as part of an ongoingbackground process. The results of this background process may beprovided to the advertiser as the result of a query when the advertisersubsequently performs or requests an analysis of the performance data.Having a cached copy of the data already available enables the requestto be processed quickly and with a relatively small amount of processingoverhead. Moreover, as similarly discussed above, the performance dataobjects may be generated based on a target time zone to providecustomized time series to the advertiser.

Method 900 may proceed to operation 908 during which it may bedetermined whether or not there are additional time parameters. Assimilarly discussed above, additional performance data objects may begenerated for different time parameters. For example, the firstperformance data object may have been generated based on a first timeparameter having a first length of ten days. A second time parameterhaving a second length of 1.5 months and a third time parameter having athird length of 3 months may also exist. Accordingly, if it isdetermined that additional time parameters exist, method 900 may returnto operation 902 and additional performance data objects may begenerated. As discussed above, a component, such as a query server orsystem, may select various intermediate data objects and group thembased on their respective data ranges, and include the selectedintermediate data objects in a performance data object which may bestored in a data storage system. If it is determined that no additionaltime parameters exist, method 900 may terminate.

FIG. 10 illustrates a data processing system configured in accordancewith some embodiments. Data processing system 1000, also referred toherein as a computer system, may be used to implement one or morecomputers or processing devices used in a controller, server, or othercomponents of systems described above, such as a data analyzer. In someembodiments, data processing system 1000 includes communicationsframework 1002, which provides communications between processor unit1004, memory 1006, persistent storage 1008, communications unit 1010,input/output (I/O) unit 1012, and display 1014. In this example,communications framework 1002 may take the form of a bus system.

Processor unit 1004 serves to execute instructions for software that maybe loaded into memory 1006. Processor unit 1004 may be a number ofprocessors, as may be included in a multi-processor core. In variousembodiments, processor unit 1004 is specifically configured to processlarge amounts of data that may be involved when processing performancedata associated with one or more advertisement campaigns, as discussedabove. Thus, processor unit 1004 may be an application specificprocessor that may be implemented as one or more application specificintegrated circuits (ASICs) within a processing system. Such specificconfiguration of processor unit 1004 may provide increased efficiencywhen processing the large amounts of data involved with the previouslydescribed systems, devices, and methods. Moreover, in some embodiments,processor unit 1004 may be include one or more reprogrammable logicdevices, such as field-programmable gate arrays (FPGAs), that may beprogrammed or specifically configured to optimally perform thepreviously described processing operations in the context of large andcomplex data sets sometimes referred to as “big data.”

Memory 1006 and persistent storage 1008 are examples of storage devices1016. A storage device is any piece of hardware that is capable ofstoring information, such as, for example, without limitation, data,program code in functional form, and/or other suitable informationeither on a temporary basis and/or a permanent basis. Storage devices1016 may also be referred to as computer readable storage devices inthese illustrative examples. Memory 1006, in these examples, may be, forexample, a random access memory or any other suitable volatile ornon-volatile storage device. Persistent storage 1008 may take variousforms, depending on the particular implementation. For example,persistent storage 1008 may contain one or more components or devices.For example, persistent storage 1008 may be a hard drive, a flashmemory, a rewritable optical disk, a rewritable magnetic tape, or somecombination of the above. The media used by persistent storage 1008 alsomay be removable. For example, a removable hard drive may be used forpersistent storage 1008.

Communications unit 1010, in these illustrative examples, provides forcommunications with other data processing systems or devices. In theseillustrative examples, communications unit 1010 is a network interfacecard.

Input/output unit 1012 allows for input and output of data with otherdevices that may be connected to data processing system 1000. Forexample, input/output unit 1012 may provide a connection for user inputthrough a keyboard, a mouse, and/or some other suitable input device.Further, input/output unit 1012 may send output to a printer. Display1014 provides a mechanism to display information to a user.

Instructions for the operating system, applications, and/or programs maybe located in storage devices 1016, which are in communication withprocessor unit 1004 through communications framework 1002. The processesof the different embodiments may be performed by processor unit 1004using computer-implemented instructions, which may be located in amemory, such as memory 1006.

These instructions are referred to as program code, computer usableprogram code, or computer readable program code that may be read andexecuted by a processor in processor unit 1004. The program code in thedifferent embodiments may be embodied on different physical or computerreadable storage media, such as memory 1006 or persistent storage 1008.

Program code 1018 is located in a functional form on computer readablemedia 1020 that is selectively removable and may be loaded onto ortransferred to data processing system 1000 for execution by processorunit 1004. Program code 1018 and computer readable media 1020 formcomputer program product 1022 in these illustrative examples. In oneexample, computer readable media 1020 may be computer readable storagemedia 1024 or computer readable signal media 1026.

In these illustrative examples, computer readable storage media 1024 isa physical or tangible storage device used to store program code 1018rather than a medium that propagates or transmits program code 1018.

Alternatively, program code 1018 may be transferred to data processingsystem 1000 using computer readable signal media 1026. Computer readablesignal media 1026 may be, for example, a propagated data signalcontaining program code 1018. For example, computer readable signalmedia 1026 may be an electromagnetic signal, an optical signal, and/orany other suitable type of signal. These signals may be transmitted overcommunications links, such as wireless communications links, opticalfiber cable, coaxial cable, a wire, and/or any other suitable type ofcommunications link.

The different components illustrated for data processing system 1000 arenot meant to provide architectural limitations to the manner in whichdifferent embodiments may be implemented. The different illustrativeembodiments may be implemented in a data processing system includingcomponents in addition to and/or in place of those illustrated for dataprocessing system 1000. Other components shown in FIG. 10 can be variedfrom the illustrative examples shown. The different embodiments may beimplemented using any hardware device or system capable of runningprogram code 1018.

Although the foregoing concepts have been described in some detail forpurposes of clarity of understanding, it will be apparent that certainchanges and modifications may be practiced within the scope of theappended claims. It should be noted that there are many alternative waysof implementing the processes, systems, and apparatus. Accordingly, thepresent examples are to be considered as illustrative and notrestrictive.

What is claimed is:
 1. A system comprising: a first processing nodeconfigured to receive a plurality of online advertising data records,the plurality of online advertising data records including a pluralityof data events characterizing a plurality of interactions between atleast one user and an online advertisement campaign, and the pluralityof online advertising data records including timestamp datacharacterizing a plurality of creation dates associated with theplurality of data events; a second processing node configured togenerate a plurality of intermediate data objects by partitioning atleast some of the plurality of online advertising data records based ona first plurality of temporal data categories, each temporal datacategory of the first plurality of temporal data categories representinga different unit of time, and each intermediate data object of theplurality of intermediate data objects being associated with a temporaldata category of the first plurality of temporal data categories; and athird processing node configured to generate a plurality of performancedata objects by partitioning the plurality of intermediate data objectsbased on a second plurality of temporal data categories, the secondplurality of temporal data categories being different than the firstplurality of temporal data categories, and each performance data objectof the plurality of performance data objects being associated with atemporal data category of the second plurality of temporal datacategories.
 2. The system of claim 1, wherein the first processing nodeis further configured to: generate a plurality of aggregate data objectsby partitioning the online advertising data records based on a thirdplurality of temporal data categories, the plurality of aggregate dataobjects being generated incrementally, and the generating of theplurality of aggregate data objects being responsive to receiving atleast some of the plurality of online advertising data records.
 3. Thesystem of claim 2, wherein the plurality of intermediate data objects isgenerated based, at least in part, on the plurality of aggregate dataobjects.
 4. The system of claim 1, wherein each intermediate data objectof the plurality of intermediate data objects includes a sequentialrepresentation of at least one online advertising data record identifiedbased on a temporal category of the first plurality of categories, andwherein the first plurality of temporal data categories includes atleast one of a day, week, and month.
 5. The system of claim 4, whereinat least one performance data object of the plurality of performancedata objects is generated by combining two or more intermediate dataobjects, and wherein the plurality of intermediate data objects and theplurality of performance data objects are stored in a data storagesystem configured to process performance data queries.
 6. The system ofclaim 5 further comprising a fourth processing node configured to:receive a query request from an advertisement campaign managementapplication program interface (API); execute the query on the pluralityof intermediate data objects and the plurality of performance dataobjects stored in the data storage system; and generate a result objectthat includes a result of the query.
 7. The system of claim 6, whereinresult object includes a combination of at least one intermediate dataobject and at least one performance data object.
 8. The system of claim7, wherein the fourth processing node is configured to: identify the atleast one intermediate data object and at least one performance dataobject included in the combination based, at least in part, on the firstand second temporal data categories associated with the at least oneintermediate data object and at least one performance data object; andinclude the at least one intermediate data object and at least oneperformance data object in the result object.
 9. The system of claim 1further comprising a fifth processing node configured to: identify aplurality of duplicative data events included in the plurality of onlineadvertising data records; and remove the plurality of duplicative dataevents from the plurality of online advertising data records.
 10. Thesystem of claim 1 further comprising a sixth processing node configuredto identify a plurality of actions based, at least in part, on theplurality of data events included in the plurality of online advertisingdata records, the plurality of actions being responsive to a pluralityof impressions included in the online advertisement campaign.
 11. Thesystem of claim 1, wherein the first processing node, the secondprocessing node, and the third processing node are pipelined.
 12. Thesystem of claim 1, wherein the plurality of intermediate data objectsand the plurality of performance data objects are stored in adistributed database system.
 13. A device comprising: a data aggregatorconfigured to receive a plurality of online advertising data records,the plurality of online advertising data records including a pluralityof data events characterizing a plurality of interactions between atleast one user and an online advertisement campaign, and the pluralityof online advertising data records including timestamp datacharacterizing a plurality of creation dates associated with theplurality of data events; an intermediate data object generatorconfigured to generate a plurality of intermediate data objects bypartitioning at least some of the plurality of online advertising datarecords based on a first plurality of temporal data categories, eachtemporal data category of the first plurality of temporal datacategories representing a different unit of time, and each intermediatedata object of the plurality of intermediate data objects beingassociated with a temporal data category of the first plurality oftemporal data categories; and a performance data object generatorconfigured to generate a plurality of performance data objects bypartitioning the plurality of intermediate data objects based on asecond plurality of temporal data categories, the second plurality oftemporal data categories being different than the first plurality oftemporal data categories, and each performance data object of theplurality of performance data objects being associated with a temporaldata category of the second plurality of temporal data categories. 14.The device of claim 13, wherein the data aggregator is furtherconfigured to: generate a plurality of aggregate data objects bypartitioning the online advertising data records based on a thirdplurality of temporal data categories, the plurality of aggregate dataobjects being generated incrementally, and the generating of theplurality of aggregate data objects being responsive to receiving atleast some of the plurality of online advertising data records.
 15. Thedevice of claim 13, wherein each intermediate data object of theplurality of intermediate data objects includes a sequentialrepresentation of at least one online advertising data record identifiedbased on a temporal category of the first plurality of categories, andwherein the first plurality of temporal data categories includes atleast one of a day, week, and month.
 16. The device of claim 15, whereinat least one performance data object of the plurality of performancedata objects is generated by combining two or more intermediate dataobjects, and wherein the plurality of intermediate data objects and theplurality of performance data objects are stored in a data storagesystem configured to process performance data queries.
 17. The device ofclaim 16 further comprising a query node configured to: receive a queryrequest from an advertisement campaign management application programinterface (API); execute the query on the plurality of intermediate dataobjects and the plurality of performance data objects stored in the datastorage system; and generate a result object that includes a result ofthe query, wherein result object includes a combination of at least oneintermediate data object and at least one performance data object. 18.The device of claim 17, wherein the query node is configured to:identify the at least one intermediate data object and at least oneperformance data object included in the combination based, at least inpart, on the first and second temporal data categories associated withthe at least one intermediate data object and at least one performancedata object; and include the at least one intermediate data object andat least one performance data object in the result object.
 19. One ormore computer readable media having instructions stored thereon forperforming a method, the method comprising: receiving a plurality ofonline advertising data records, the plurality of online advertisingdata records including a plurality of data events characterizing aplurality of interactions between at least one user and an onlineadvertisement campaign, and the plurality of online advertising datarecords including timestamp data characterizing a plurality of creationdates associated with the plurality of data events; generating aplurality of intermediate data objects by partitioning at least some ofthe plurality of online advertising data records based on a firstplurality of temporal data categories, each temporal data category ofthe first plurality of temporal data categories representing a differentunit of time, and each intermediate data object of the plurality ofintermediate data objects being associated with a temporal data categoryof the first plurality of temporal data categories; and generating aplurality of performance data objects by partitioning the plurality ofintermediate data objects based on a second plurality of temporal datacategories, the second plurality of temporal data categories beingdifferent than the first plurality of temporal data categories, and eachperformance data object of the plurality of performance data objectsbeing associated with a temporal data category of the second pluralityof temporal data categories.
 20. The one or more computer readable mediarecited in claim 19, wherein the method further comprises: receiving aquery request from an advertisement campaign management applicationprogram interface (API); executing the query on the plurality ofintermediate data objects and the plurality of performance data objects;and generating a result object that includes a result of the query,wherein result object includes a combination of at least oneintermediate data object and at least one performance data object.